Natural Video Matting with Depth

Video matting is the process of taking a sequence of frames, isolating the foreground, and replacing the background with something different in each frame. This is an under-constrained problem when the background is unknown. Matting techniques exist to approximate these values using manual input cues. We look at existing singleframe matting techniques and present a method that improves upon them by adding depth information acquired by a time-of-flight range scanner. We use the depth information to automate the process so it can be practically used for video sequences. In addition, we show that we can improve the results from natural matting algorithms by adding a depth channel. The additional depth information allows us to reduce the artifacts that arise from ambiguities that occur when an object is a similar color to its background. Figure 1. Natural matting. From left to right, top to bottom: The original image, a trimap, alpha matte, foreground composed on a new background.